Method for interactive segmentation of a structure contained in an object

ABSTRACT

The invention relates to a method for the interactive segmentation of a structure contained in an object from a three-dimensional image of the object. In an expansion mode a region expansion process is carried out, the results of said process simply being completely or partly undone again in a contraction mode. A quasi-continuous similarity measure can be used to determine whether or not a voxel belongs to the structure to be segmented. The expansion taking place in the region expansion process always involves the voxels having the largest similarity measure each time.

The invention relates to a method for the interactive segmentation of astructure contained in an object from a three-dimensional image of theobject. The invention also relates to an image processing device forcarrying out the method, to a computer program for such an imageprocessing unit, and to an imaging device including such an imageprocessing unit.

Known methods for the segmentation of a structure contained in an objectare based on a so-called region growing or region expansion process inwhich the user defines one or more so-called seed voxels in thestructure to be segmented. Subsequently, the voxels neighboring the seedvoxels are examined as to whether or not they belong to the structure.The voxels belonging to the structure form the first generation ofvoxels whose neighboring voxels are subsequently examined as to whetheror not they belong to the structure. If they belong to the structure,they form the second generation of voxels whose neighboring voxels againare examined as to whether or not they belong to the structure, and soon. The voxels are thus examined one generation after the other untilthe process is stopped automatically if no further voxels belonging tothe structure are found.

The quality of the segmentation achieved in this manner is dependent toa decisive degree on the quality of the similarity criterion whichgoverns the decision as to whether or not a voxel belongs to thestructure to be segmented. It may occur that voxels are assigned to thestructure even though they do not belong to the structure, or thatvoxels which belong to the structure are not assigned to the structure,or both. It may be necessary for the user to repeat the segmentationmethod while using a modified similarity criterion.

EP 0516047 discloses a method of the kind set forth in which the user isinteractively engaged in the segmentation process. Therein, theinstantaneous segmentation status is continuously reproduced in asegmentation image while utilizing the voxels which have been recognizedby the method as belonging to the structure. When the user detects thatthe segmentation image also shows a region which, on the basis of theuser's knowledge of the anatomy, cannot belong to the structuresearched, the user defines a seed voxel in this region.

As from this seed voxel, a new expansion process is started whose resultis superposed on the segmentation image, for example, in a differentcolor. This expansion process first determines voxels from the regionwhich does not belong to the structure. When this second expansionreaches the structure to be segmented, the user can interrupt thissecond expansion and erase the region (not belonging to the structure)thus determined, so that only the structure searched remains as thesegmentation result.

For this interactive segmentation to be successful it is a prerequisitethat there is only one region (or only very few regions) in which the(first) expansion can take place and that these regions are connected tothe structure only via one (or very few) as narrow as possibleconnecting links. When the structure is enclosed by a large regionhaving characteristics similar to those of the structure, it ispractically impossible for the user to separate this region from thestructure within reasonable limits of cost and effort.

It is an object of the present invention to provide an improved methodfor the interactive segmentation of a structure.

In accordance with the invention this object is achieved by means of amethod for the interactive segmentation of an object contained in anobject from a three-dimensional image of the object, which methodinvolves a continuous visualization of the instantaneous segmentationstatus in a segmentation image and includes an expansion mode and acontraction mode, the following steps being carried out in the expansionmode:

-   a). determining the similarity between the neighboring voxel of a    defined voxel and the structure,-   b). storing the order in which the similarity of the voxels    belonging to the structure is determined,-   c). repeating the steps a) and b) for all voxels of the structure    which have not yet been processed or have been newly determined,    while in the contraction mode the voxels are removed, on the basis    of the instantaneous segmentation status, from the segmented    structure in an order which is the reverse of that in which they    have been determined as belonging to the structure in the expansion    mode.

The invention offers the user a choice between the expansion mode andthe contraction mode after the start of the segmentation. When thesegmentation has not yet progressed far enough, the user will choose theexpansion mode and hence continue the region growing process. However,if the segmentation has progressed too far, so that the segmentation hasalready reached regions which no longer belong to the structure, theuser will choose the contraction mode in which the segmentation isundone one voxel after the other, so that the previous stages of thesegmentation can be reached again.

The best segmentation possible for the defined assignment of voxels tothe structure can thus be achieved in a simple manner. If thisassignment is binary (for example, 1=the voxel belongs to the structure,0=the voxel does not belong to the structure), the quality of thesegmentation will be dependent on the degree of correctness of thisassignment.

This dependency is eliminated in the version disclosed in claim 2. Forthe evaluation of the neighboring voxels a quasi-continuous similaritymeasure is then determined instead of a binary similarity criterion;this quasi-continuous similarity measure describes the extent to whichthe voxel being examined has a property which is characteristic of thestructure. For example, if a bone structure in a CT image ischaracterized in that it exhibits the highest Hounsfield values, it isnot necessary to determine whether the Hounsfield value of a voxel islarge enough so as to belong to the structure or not. It sufficesinstead to use the Hounsfield value (which may be between −1000 and+1000) as the similarity measure.

In this respect it is essential that the selection of the voxels whoseneighbors are to be examined does not take place in generations, but onthe basis of the value of the relevant similarity measure. Therefore,this method initially expands only in the regions whose characteristicsare most similar to those of the structure. In the course of the furtherprocess, those regions are added in which the characteristics of thestructure are less pronounced. Should the user not interrupt theexpansion process at all, the result could be the segmentation of animage which corresponds to the three-dimensional image to be segmented.However, if the user stops the expansion in time, a very goodsegmentation image will be obtained.

The version disclosed in claim 3 makes it possible for the user todefine in advance how far the expansion (or the reversal of theexpansion) should proceed before it is automatically stopped. The numberof selection steps can then indicate how many steps should be carriedout during the expansion starting at the seed voxel, but also a numberof steps which is based on the instantaneous segmentation status. Whenthis choice is made after an interruption of the segmentation by theuser, the user's choice determines whether subsequently the expansionmode or the contraction mode takes place.

A very large number of voxels can be stored in the list in the course ofthe segmentation method. The selection of the voxel having the highestsimilarity measure from the list would then require a comparatively longperiod of time, even if the similarity measure were stored therein inaddition to the position of the voxel. This search is significantlysimplified by the version disclosed in claim 4, because the search cancommence with the sub-list with which the highest value of thesimilarity measure is associated and finish as soon as a voxel notpreviously selected is encountered in this sub-list and the next-highersub-lists. The search becomes even simpler when each time the sub-listin which such a voxel is present is marked during the segmentationprocess.

In conformity with the version disclosed in claim 5, neighboring voxelshaving a low similarity measure are not even stored in the list, so thatthe list becomes less extensive. If the minimum value was chosen to betoo small, as opposed to the known methods such a choice would not havea negative effect on the quality of the segmentation result. When theselected voxel is situated at the edge of a structure, in conformitywith this version it may occur that none of its neighboring voxels istaken up in the list (apart from the voxel as from which the relevantvoxel was determined during a preceding expansion step).

The version of the invention disclosed in claim 6 is particularlysuitable for the segmentation of bone structures in a CT image. In thesimplest case the similarity measure of a voxel then corresponds to itsgrey value (that is, the Hounsfield value).

Claim 7 describes an image processing device for carrying out the methodin accordance with the invention while claim 8 discloses a computerprogram for such an image processing device and claim 9 discloses adiagnostic medical imaging device, notably a computed tomographyapparatus including such an image processing device.

The invention will be described in detail hereinafter with reference tothe drawings. Therein:

FIG. 1 shows an image processing unit for carrying out the method inaccordance with the invention,

FIG. 2 shows a flow chart of an imaging method up to the segmentation,

FIG. 3 a shows a first part of a detailed flow chart,

FIG. 3 b shows a second part of such a detailed flow chart,

FIG. 3 c shows a third part of such a detailed flow chart, and

FIG. 4 shows the structure of a list used in this context.

The image processing device shown in FIG. 1 includes an image processingand control processor 1 which is provided with a memory 2 which iscapable of storing a three-dimensional image of an object to be examinedand also some lists required for the execution of the method. The imageprocessing and control processor 1 is connected, via a bus system 3, toan imaging device 4 which is shown purely diagrammatically, for example,a computed tomography apparatus or an MR apparatus. The results obtainedby means of the invention can be displayed on a monitor 6. The user canaccess the image processing and control processor 1 via a keyboard orother input units which are not so shown in the drawing, thus exertingan effect on the execution of the segmentation process.

FIG. 2 is a diagrammatic representation of a procedure which commenceswith the data acquisition for a three-dimensional image of an object andends with the segmentation of a structure within this object. Using acomputed tomography apparatus, CT data of an object is acquired in step101. In step 102 an image of the object is reconstructed from this CTdata. Subsequently, in step 103 the user can start the expansion mode inorder to segment a structure on the basis of the region growing processin accordance with the invention. The relevant segmentation status iscontinuously displayed for the user, for example, a physician, in asegmentation image (step 104) which is continuously updated, so that theuser can watch the growth of the segmented structure.

The user has many possibilities for intervention. The user can interruptthe segmentation process or terminate it (in step 105). After theinterruption, the user can continue the expansion mode. However, theuser can also reverse the segmentation process if the segmentation hasproceeded too far, said reversal being initiated by activating thecontraction mode. In block 106 the expansion steps are then undone againin the reverse order, so that the segmentation is set back to an earlierphase of the expansion again.

FIGS. 3 a, 3 b and 3 c show details of the segmentation method. Afterthe initialization, the user defines at least one seed voxel in thethree-dimensional CT image, that is, a voxel situated at the center ofthe structure to be segmented (step 201). The user may also define aplurality of seed voxels; this is necessary, for example, when twostructures which are separated from one another in space (for example,the two shoulder blades) are to be segmented in the CT image. It is alsopossible to define different types of seed voxels so as to enablediscrimination of different structures of similar consistency, forexample, the shoulder blades from the vertebral column. Moreover, theuser can set further input parameters in the step 201.

Subsequently, in step 202 the seed voxels and their similarity measureare stored in a first list which will be referred to as the expansionlist hereinafter. The similarity measure of a voxel defines the degreeto which the voxel exhibits a property which is characteristic of therelevant structure, that is, quasi the probability that a voxel belongsto the structure. In the simplest case the similarity measure may bedefined by the grey value of the relevant voxel in the three-dimensionalimage, that is, by the Hounsfield value in the case of a CT image. Thissimple similarity measure is suitable for the segmentation of bonestructures in a CT image, because they have the highest Hounsfieldvalues. When other structures are to be segmented, other similaritymeasures must be used.

The number of expansion steps is defined in step 203. In this respect itis effective to specify a very large number (for example, an infinitenumber), so that the segmentation process could be continued until it isfinished, that is, if it is not terminated by the user before that. Thesetting can be performed graphically, for example, by means of a sliderwhich is displayed on the monitor 6 and can be shifted by the user. Theinstantaneous number of steps can be indicated by means of a bar whichis situated underneath the slider and whose length can vary inconformity with the number of expansion steps; the target has been metwhen the bar reaches the slider.

In the step 204 it is checked whether the defined number of expansionsteps is larger than the number of expansion steps carried out thus far.If this is not the case, a contraction mode (FIG. 3 b) is activated, andotherwise the expansion mode (FIG. 3 c). After the start, in the step204 first the expansion mode is activated. This mode comprises a loopwhich is defined by blocks 205 and 211 and is completed a number oftimes until the defined number of steps is reached or the expansion listis empty or the user stops the expansion before that. In step 206 thevoxel having the highest similarity measure is selected from theexpansion list. When the loop is completed the first time, this voxel isalways one of the seed voxels. When the loop is completed for the secondtime, however, a voxel which does not belong to the seed voxels may alsobe selected, that is, if this voxel has a similarity measure which ishigher than that of the remaining seed voxels.

The individual voxels and their similarity measure could in principle bestored in the expansion list. Considering the large number of voxelswhich could occur in the course of the expansion process, too muchstorage space would then be required and a large amount of calculationtime would be necessary so as to find each time the voxel having thehighest similarity measure.

FIG. 4 is a diagrammatic representation of a form of the expansion listwhich requires less storage space and enables faster processing. Theexpansion list is now subdivided into as many sub-lists 300, 301 . . .302, 303 and 304 as there may be Hounsfield values present in the CTimage, so that each sub-list is associated with one of the Hounsfieldvalues; these values are stated in the column of boxes shown at the leftin FIG. 4. The boxes, constituting each sub-list, symbolize the elementsof the sub-list which can take up one voxel each. A white (empty)element means that no voxel is stored therein as yet. In each sub-listthere is marked in black the element which takes up each time the nextvoxel belonging to this sub-list (or to this similarity measure). To theleft thereof there are indicated, by way of hatching, the elements whichcontain voxels which have not been selected thus far. Voxels which havealready been selected during the expansion process are denoted bycross-hatching to the left thereof. Some of these voxels are visible inthe segmentation image (these voxels are provided with dots in additionto the hatching) and others are not.

Each element of the sub-lists can store, for example, 32 bits: 29 bitsfor the position of the voxel, 2 bits which characterize the type ofseed voxel wherefrom the relevant voxel was derived during theexpansion, and 1 bit which indicates whether the relevant voxel isvisible in the instantaneous segmentation image or not. Moreover, foreach sub-list there is provided a pointer Z which indicates the elementwhich is selected as the next one in its sub-list. Finally, allsub-lists which contain a voxel not selected thus far will have beenmarked (not shown).

In the step 206, after the selection of the voxel (in the case of theexpansion list shown in FIG. 4 this would be the voxel from the sub-list303 pointed out by the pointer Z), the pointer in the sub-list isshifted one position to the right, and the counter for the number ofexpansion steps is incremented by 1. Moreover, the similarity measure ofthis voxel is taken up in a second list which will be referred to as thecontraction list hereinafter.

Subsequently, there is formed a further loop which is defined by blocks207 and 210 and is completed six times, that is, once for each voxelwhich neighbors the selected voxel by way of one side. In step 209 it istested whether or not the relevant voxel satisfies a similaritycriterion possibly set in the step 201. In the present example thiswould mean that it is checked whether the Hounsfield value of the voxelreaches a minimum value or not. However, it is not necessary for theuser to set such a similarity criterion. If no criterion is defined, allvoxels not yet taken up in the expansion list are then taken up in thesub-list associated with their respective similarity measure. Theneighboring voxels then “inherit” the characterization of the seed voxelfrom the expansion of which they have emanated.

When the user has set seed voxels on the shoulder blades, the collarbones and the vertebral column in the step 202, in the expansion modethe shoulder blades and the collar bones will become visible first inthe continuously updated segmentation image, because the seed voxelssituated therein, and their neighboring voxels, have a Hounsfield valuewhich is higher than that of the seed voxel set on the vertebral column.The vertebral column will start to “grow” only after the growing of theother two regions has terminated. The vertebra on which a seed voxel hasbeen set then starts to grow first. It is only after this vertebra hasbeen completed in the segmentation image that the expansion can proceedto the neighboring vertebra via the disc situated therebetween. The ribsbecome successively visible in the segmentation image only after theentire vertebral column has become visible, said ribs being connected tothe vertebral column via a cartilaginous mass whose Hounsfield valuesare lower than those of the ribs or the vertebrae.

Therefore, it is important that the user (provided that the user sets asimilarity criterion in the form of a minimum value of the similaritymeasure which is tested in the step 209) sets this minimum value to beso small that not only the bone structures are presented to the user,but also the cartilaginous structures situated therebetween. As soon asthe expansion reaches a rib, the expansion of the path through thecartilaginous mass stops and the expansion in the bony rib commences. Asa result, the intermediate spaces between rib and vertebral columnremain to a high degree recognizable as such in the segmentation image.

The expansion mode stops in step 212 when the defined number of steps isreached or when the expansion list contains only voxels which havealready been selected in the step 206 and subsequently processed, orwhen the user intervenes interactively. The user can then change theparameters of the visualization. For example, the user can change theviewing direction (in the case of a rendering of the surface), so thatthe user obtains a different view of the structure and voxels which werenot visible thus far become visible and vice versa. If in the simplestcase the segmentation image is formed by a slice image, the user canalso change the position of the slice shown.

When the user considers the segmentation status reached to be adequate,the user can terminate the process. If the expansion process executedthus far has not proceeded far enough or has gone too far, the user canchange the number of expansion steps in step 213 (FIG. 3 a). If theexpansion has not gone far enough, the user will set a larger number ofexpansion steps in the step 213 and/or lower the similarity criteriontested in the step 209, after which the expansion process with the steps205 to 211 is carried out again.

When the expansion has gone too far in the opinion of the user, the userwill set a smaller number of expansion steps in the step 213.Subsequently, a contraction mode is activated which reverses thesegmentation, the expansion steps last executed then being undone first.The execution of the contraction process is thus analogous to that ofthe expansion process, be it in the reverse direction. It includes afirst loop with the elements 305 and 311, which loop is completed againand again until a segmentation status is obtained which corresponds tothe newly set (smaller) number of expansion steps or until thecontraction list is empty (in this case the segmentation status prior tothe beginning of the expansion method would have been reached again).Additionally, the user can interrupt the execution of the loop when heconsiders the segmentation status to be adequate.

In step 306 the step counter is decremented each time by 1, and the lastvoxel taken up in the contraction list is removed therefrom again. Thecomposition of the contraction list could be analogous to that of theexpansion list. However, it is simpler to store merely the similaritymeasure of the selected voxel in the contraction list in the step 206.In the contraction mode the associated voxel is then obtained from thesub-list associated with this similarity measure, for example, 303, andthe position of the pointer in this sub-list. In the step 306 thesimilarity measure last stored in the contraction list is thus erasedand the pointer Z in the sub-list provided for this similarity measureis shifted one position further to the left.

In the loop defined by the elements 307 and 310 the neighboring voxelsare subsequently tested. When it is determined in the step 309 that therelevant neighboring voxel was taken up in the expansion list, it isremoved therefrom again in the step 308. In this contraction process theexpansion is thus undone one step after the other. The contractionprocess ends when the loop 305 . . . 311 has been completed an adequatenumber of times or when the user stops the contraction. The process thenproceeds to the step 212. The user then again has the possibility ofterminating the segmentation process, of continuing the contractionprocess or of (partly) undoing the results of the preceding contractionprocess again.

The invention can in principle be used for the segmentation of otherstructures having a different similarity measure. For example, for thesegmentation of soft tissues it may be necessary to segment regionswhich are as homogeneous as possible. In that case the similaritymeasure is an as small as possible difference between the grey value ofthe voxel being examined and the gray value of the seed voxel.

In another method for the segmentation of carcinogous tissue, regions ofthe same texture (the type of the gray value structuring) are recognizedas belonging to the structure. Accordingly, a suitable similaritymeasure in this respect would take into account the similarity of thegray value distribution in a vicinity of the voxel being examined.

1. A method for the interactive segmentation of a structure contained inan object from a three-dimensional image of the object, which methodinvolves a continuous visualization of the instantaneous segmentationstatus in a segmentation image and includes an expansion mode and acontraction mode, the following steps being carried out in the expansionmode a) determining the similarity between the neighboring voxel of adefined voxel and the structure, b) storing the order in which thesimilarity of the voxels belonging to the structure is determined, c)repeating the steps a) and b) for voxels of the structure which have notyet been processed or have been newly determined, while in thecontraction mode the voxels are removed, on the basis of theinstantaneous segmentation status, from the segmented structure in anorder which is the reverse of that in which they have been determined asbelonging to the structure in the expansion mode.
 2. A method as claimedin claim 1, in which the expansion mode includes the following steps a)determining a similarity measure, describing the similarity between avoxel and the structure, for the neighboring voxel of the set voxel ofthe structure, b) storing the neighboring voxel in a list, c) selectingthe voxel having the largest similarity measure from the list, d)determining the similarity measure for the neighboring voxel of theselected voxel, e) storing neighboring voxels in the list, f) repeatingthe steps f) to h) for voxels which have not been selected thus far. 3.A method as claimed in claim 2, in which the number of selection stepscan be interactively set and the segmentation, or the reversal of thesegmentation, is continued until a segmentation status corresponding tothe number of selection steps is reached.
 4. A method as claimed inclaim 2, in which the list is subdivided into a number of sub-lists (300. . . 304) in which voxels having the same value of the similaritymeasure are taken up, each of said sub-lists being associated with adifferent value of the similarity measure.
 5. A method as claimed inclaim 2, in which from among the voxels neighboring a voxel selectedfrom the list exclusively those voxels whose similarity measure exceedsa minimum value are taken up in the list.
 6. A method as claimed inclaim 2, in which the similarity measure of the voxels is derived fromtheir grey value.
 7. An image processing device for carrying out themethod claimed in claim 1, which device includes a memory for storing athree-dimensional image of the object and for storing lists which areprocessed in the course of the segmentation, an image display unit fordisplaying a segmented structure, and image processing means forsegmenting a three-dimensional structure which is contained in theobject, said image processing means having an expansion mode and acontraction mode, the following steps being carried out in the expansionmode: a) determining the similarity between the neighboring voxel of adefined voxel and the structure, b) storing the order in which thesimilarity of the voxels belonging to the structure is determined, c)repeating the steps a) and b) for voxels of the structure which have notyet been processed or have been newly determined, while in thecontraction mode the voxels are removed, on the basis of theinstantaneous segmentation status, from the segmented structure in anorder which is the reverse of that in which they have been determined asbelonging to the structure in the expansion mode.
 8. A computer programfor an image processing unit as claimed in claim 7 for the segmentationof a three-dimensional structure, contained in an object, from athree-dimensional image of the object, which program includes the stepsof: a) determining the similarity between the neighboring voxel of adefined voxel and the structure, b) storing the order in which thesimilarity of the voxels belonging to the structure is determined, c)repeating the steps a) and b) for voxels of the structure which have notyet been processed or have been newly determined, while in thecontraction mode the voxels are removed, on the basis of theinstantaneous segmentation status, from the segmented structure in anorder which is the reverse of that in which they have been determined asbelonging to the structure in the expansion mode.